[miniflare] Add backend resources for email capture and storage - #15064
[miniflare] Add backend resources for email capture and storage#15064tpmmorris wants to merge 10 commits into
Conversation
🦋 Changeset detectedLatest commit: 1291f71 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
| rawBase64: bytesToBase64(rawEmailBuffer), | ||
| }); | ||
|
|
||
| this.ctx.waitUntil( |
There was a problem hiding this comment.
🟡 Emails sent just before the dev session shuts down are never written to disk or logged
The on-disk copy of a sent email and its log line are queued to run in the background (this.ctx.waitUntil(...) at packages/miniflare/src/workers/email/send_email.worker.ts:370 and :451) instead of being finished before the send call returns, so a script that sends an email and then immediately shuts the local dev session down loses the saved message entirely.
Impact: Short-lived usages (for example sending through getPlatformProxy() and then disposing) no longer reliably produce the .eml/text/HTML/attachment files or the "send_email binding called..." log they used to.
Why the deferred work can be dropped
Before this change send() awaited every storeTempFile() call and logged before resolving, so by the time the caller's await env.SEND_EMAIL.send(...) returned the files existed. Now both branches resolve immediately after the in-workerd capture, deferring the loopback /core/store-temp-file writes and logging to ctx.waitUntil.
Miniflare#dispose() aborts, stops the loopback server and tears down workerd before drainEmailArtifactManager() runs (packages/miniflare/src/index.ts:3490-3496); drain() only awaits operations that already reached the Node side (packages/miniflare/src/plugins/email/artifacts.ts:86-89), so waitUntil work that has not yet issued its loopback request is simply discarded.
The test updates in this PR reflect the new asynchrony (the miniflare email specs now poll with vi.waitFor, and the get-platform-proxy e2e no longer asserts on the file), but callers that dispose right after sending have no way to wait.
Was this helpful? React with 👍 or 👎 to provide feedback.
@cloudflare/autoconfig
@cloudflare/build-output-utils
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-functions
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-pool-workers
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
7fccbb7 to
494e214
Compare
petebacondarwin
left a comment
There was a problem hiding this comment.
Please address the bugs highlighted by Devin
emily-shen
left a comment
There was a problem hiding this comment.
just a quick first pass on the API surface
| "tags": ["Local Explorer"] | ||
| } | ||
| }, | ||
| "/email/routing": { |
There was a problem hiding this comment.
so because these have different semantics to the actual endpoints, we need to stick these in the local namespace. otherwise it will cause confusion between the 'real' api and this one
(https://developers.cloudflare.com/api/resources/email_routing/methods/get)
| "schema": { | ||
| "type": "string" | ||
| }, | ||
| "description": "Deliver the test email to this worker's email() handler, regardless of address-based routing." |
There was a problem hiding this comment.
what do you mean by address-based routing? the port? that won't hold because there can be multiple workers per port. you could either use the worker name as part of the path or make this required
| }, | ||
| "outcome": { | ||
| "type": "string", | ||
| "enum": ["ok", "exception"], |
There was a problem hiding this comment.
hmmm. as in you would want a 200 exception if the handler intentionally threw?
task failed successfully i guess 😅
There was a problem hiding this comment.
Yeah this is a weird situation because the message was sent successfully, just the worker itself didn't like it. This is a confusing way to express it but I felt using a non 200 status would make it seem like the send itself failed. I'll make it clearer in the docs whats actually happening, unless youd have a different preference to how its handled?
| "tags": ["Email"] | ||
| } | ||
| }, | ||
| "/email/routing/{email_id}": { |
There was a problem hiding this comment.
could this be a query param on GET /email/routing above
There was a problem hiding this comment.
The list and detail items have a different shape so i thought it would be appropriate to keep separate, if not I can make this change?
| "tags": ["Email"] | ||
| } | ||
| }, | ||
| "/email/sending/{email_id}": { |
There was a problem hiding this comment.
similarly to above, query param on the main list endpoint?
| "properties": { | ||
| "type": { | ||
| "type": "string", | ||
| "enum": ["received", "forward", "reply", "reject", "unhandled"], | ||
| "description": "The kind of event." | ||
| }, | ||
| "timestamp": { | ||
| "type": "string", | ||
| "description": "ISO 8601 timestamp of when the event occurred." | ||
| }, | ||
| "messageId": { | ||
| "type": "string", | ||
| "description": "Present on `forward`/`reply` events; correlates with the matching `forwards`/`replies` entry." | ||
| } |
There was a problem hiding this comment.
this could be a discriminated union to show that messageId is only available when the type is forward or reply
| }, | ||
| "headers": { | ||
| "type": "array", | ||
| "description": "Headers added to the forwarded message, as [key, value] pairs.", |
There was a problem hiding this comment.
why not an array of objects?
| description: | ||
| "Worker whose email() handler processed the message, if known.", | ||
| }, | ||
| from: { type: "string", description: "Envelope MAIL FROM address." }, |
There was a problem hiding this comment.
there's quite a lot of repetition in these objects with from/to/subject/messageId etc.
could you define an object like 'base_email' and extend that?
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
…torage and api exposure.
| { | ||
| name: CoreBindings.TEXT_FALLBACK_WORKER_NAME, | ||
| json: JSON.stringify(fallbackWorkerPublicName ?? ""), | ||
| }, |
There was a problem hiding this comment.
🟡 Fallback worker name setting is registered twice on the dev server, creating a conflicting duplicate
The entry worker's fallback-worker-name value is added a second time (serviceEntryBindings.push with json: JSON.stringify(fallbackWorkerPublicName ?? "") at packages/miniflare/src/plugins/core/index.ts:799-802) even though the same setting is already provided later in the same list, so the dev server ends up with two conflicting entries for one name.
Impact: The local dev server may refuse to start or silently pick the wrong value for which worker handles unrouted requests and received emails.
Duplicate MINIFLARE_FALLBACK_WORKER_NAME binding in serviceEntryBindings
getGlobalServices builds serviceEntryBindings for the SERVICE_ENTRY worker. The PR adds a new binding named CoreBindings.TEXT_FALLBACK_WORKER_NAME ("MINIFLARE_FALLBACK_WORKER_NAME") at packages/miniflare/src/plugins/core/index.ts:799-802 using a json value. However the pre-existing code already pushes a binding with the identical name at packages/miniflare/src/plugins/core/index.ts:928-931 using a text value (workerNames[0] ?? ""). Both live in the same serviceEntryBindings array, so the entry worker now declares two bindings with the same name. The entry worker Env type also now declares [CoreBindings.TEXT_FALLBACK_WORKER_NAME]: string twice (packages/miniflare/src/workers/core/entry.worker.ts:20 and :29), confirming the author did not notice the binding already existed. fallbackWorkerPublicName is set to this.#workerOpts[0].config.name (packages/miniflare/src/index.ts:2167), which equals workerNames[0], so the values coincide; the defect is the duplicated binding itself. The value feeds routeTarget in getTargetService (packages/miniflare/src/workers/core/entry.worker.ts:166), which is passed as workerName to handleEmail and used for the per-worker Access blob lookup. The correct fix is to remove one of the two bindings (keep a single definition).
Prompt for agents
In getGlobalServices (packages/miniflare/src/plugins/core/index.ts), the binding CoreBindings.TEXT_FALLBACK_WORKER_NAME is now pushed twice into serviceEntryBindings: once as a json binding near line 799-802 (value fallbackWorkerPublicName) and once as a text binding near line 928-931 (value workerNames[0]). This creates a duplicate binding name on the SERVICE_ENTRY worker. Consolidate these into a single binding definition (they carry the same first-worker name), and remove the now-redundant duplicate type entry in packages/miniflare/src/workers/core/entry.worker.ts (TEXT_FALLBACK_WORKER_NAME is declared twice in the Env type). Ensure the retained binding provides the value the entry worker expects for routeTarget/Access-blob lookup and the email workerName.
Was this helpful? React with 👍 or 👎 to provide feedback.
Add backend resources for Email interaction within Local Explorer
Fixes #13648
Adds storage and capture methods for emails sent from/received by a worker using durable objects for storage (in line with the new 'Observability' tab), and cdn-cgi endpoints to mimic the sending of an email to a worker. Also records actions taken by the
email()handler (received,forwarded,replied,rejected,unhandled), so that they can be mapped and displayed in local explorer in a similar manner as the dash. The Email result interface has been updated to also include a list of events.